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ABSTRACT 

The Test of Pragmatic Language (TOPL) is an individually 
administered instrument designed to assess pragmatic language skills that can 
be used with students in kindergarten through high school . It is more 
specifically intended for use with children, adolescents, and adults with 
learning disabilities, language delays, reading difficulties, or aphasia. It 
is said to provide information on six dimensions of pragmatic language: 
physical setting, audience, topic, purpose, visual -gestural cues, and 
abstraction. The 44-item test is accompanied by an examiner's manual, scoring 
materials, and the test-picture book. The normative sample consisted of 1,016 
examinees from the United States (24 states) and Canada (1 province) , 
representative with regard to sex, residence, race/ethnicity, geographic 
region, and age. Overall validity findings about the test have been 
acceptable, although limited by small sample size and some lack of 
information about student age or grade level. Previous reviews have generally 
supported the usefulness of the TOPL, although two reviewers have suggested 
that more information is needed to relate actual test items to the defined 
six areas, and one reviewed has pointed out some limitations in the 
applicability of the TOPL pictures to all students. The TOPL appears to be a 
tool to provide a profile of pragmatic or social language skills . (Contains 
five references.) (SLD) 
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General Information 

Test Title : Test of Pragmatic Language (TOPL) 

Authors : Diana Phelps-Terasaki and Trisha Phelps-Gunn 
Publisher : Pro-ed 
Publication Date : 1992 

Time Required for Administration : 30-45 minutes 

Cost : $84 per complete kit, $29 per examiner’s manual, $29 per TOPL picture book, 
$29 per 25 TOPL profile/examiner record forms. 

Brief Description of Purpose and Nature of Test 
The TOPL is an individually administered instrument designed to assess 
pragmatic language skills. The test can be used with students from kindergarten 
through high school age. More specifically, it is intended for use with children, 
adolescents and adults with learning disabilities, language delays or disorders, reading 
difficulties and adult aphasic populations. It is also designed for use with children and 
adolescents in ESL programs as well as family therapy or substance abuse treatment 
programs. The TOPL is a verbal measure which is said to provide information on six 
dimensions of pragmatic language: physical setting, audience, topic, purpose (speech 
acts), visual-gestural cues, and abstraction. The test can be used as part of a battery of 
psychological tests to assist in a mote comprehensive assessment of social skills and 
language use. The manual reports that the TOPL has 3 major purposes: to identify 
students who are below their peers in pragmatic language skills, to determine strengths 
and weaknesses of an individual’s pragmatic language skills, and to documents students 
progress as a result of an intervention. The TOPL consists of 44 items, many of which 
have corresponding pictures, which require a verbal response from the examiner. The 
pictures are black and white drawings depicting various activities in a number of 



O 

ERIC 



3 



TOPL Review 



3 



settings. Examinee’s responses are scored as 1-acceptable or 0-unacceptable. Samples 
of each are provided on the Profile/Examiner Record Form. Test time is generally 30 
to 45 minutes, but testing can be extended over several sessions for individuals with 
more severe language difficulties. The TOPL yields one score which is easily 
converted to a percentile rank, quotient score, and age equivalent score. 

Practical Evaluation 

The TOPL materials are packaged in an easily transportable carrying case which 
includes the examiner’s manual, profile/examiner record form (used for scoring) and 
the picture book. The 45 page technical manual is well-organized with clear directions 
for administration and scoring of the test. In addition, it provides detailed information 
on norms, validity and reliabUity. As mentioned previously, the picture book is 
composed of black and white pictures illustrating a variety of scenarios. However, 
there is no picture stand incorporated into the design of the book, nor is one provided. 
Thus, the examiner will have to determine the best way to present the picture to the 
student without interfering with their abiUty to score the examinee’s responses. The 
Profile/Examiner Record Form is used to record scores and other notations related to 
student responses. It explains each item including which picture to show, what to say, 
and how to score the re^nse. 

Directions in both the manual and record form are easy to follow; however, 
they faUed to include guidelines or a script for an introduction. Without guidelines for 
an introduction each examiner is free to decide what information to give the cUent 

about the purpose and nature of the test. The lack of uniform guidelines for an 

introduction could lead to greater variability in the administration of the test. 
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The TOPL can be administered by anyone who is competent in the 
administration of tests in education, speech-language pathology, or psychology (Phelps- 
Terasaki & Phelps-Gunn, 1992). The authors suggest that the examiner become 
familiar with the manual and practice administering the test before a formal assessment. 
A qualified and well prepared examiner is especially important when assessment is used 
in part to determine educational placement (i.e. special education). 

The scoring procedures for the TOPL are very straightforward. The raw score 
is the total number of correct responses provided by the test-taker. To ensure more 
accurate scoring the examiner may want to utilize a tape recorder to allow for greater 
evaluation of students responses. With careful attention to directions and a well- 
prepared examiner, administration of the TOPL is fairly easy. 

Technical Evaluation 

The normative sample is representative of the national population with 1,016 
examinees from 24 states and one Canadian providence. The sample was representative 
with regard to sex, residence, race/ethnicity, geographic region, and age. A better 
sample would have addressed socioeconomic level and relevant personality 
characteristics (Anastasi, 1988). Normative scores are reported as quotients (with 
mean of 1(X) and standard deviation of IS), percentiles, and age equivalents at each 6 
month interval. The authors realized the importance of converting raw scores into 
various forms; thus procedures for converting scores were included in the manual. The 
authors caution that age equivalents are easily misinterpreted and should only be used 
when federal or state departments require them (Phelps-Terasaki & Phelps-Gunn, 

1992). Norms are provided for students age 5-13. A primary concern of this review 
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was the authors suggestion that the TOPL is appropriate for use with high school age 
students and adults despite the lack of norms provided for those age groups. After 
careful examination of the manual, it appears the authors intended the TOPL to be used 
to assess the strengths and weaknesses of older persons language skills rather than 
evaluating their performance utilizing the normative data. This does appear to be 
viable. 

The manual provides information on reliability through internal consistency, 
interscorer reliability, and the standard error of measurement. All internal consistency 
coefficients were above .80 except at age 6 which was .74. The average coefficient 
was .82 which is acceptable for test users to interpret results. The interscorer 
reliability is much more acceptable with a coefficient of .99. The average standard 
error of measurement is 6.3 on the quotient scale, with a range from 5.0 to 7.6. 

With respect to ^i^alidity, the manual provides information on content, criterion- 
related, and construct validity. For content validity the authors point to item selection 
and item analysis procedures described in a previous section of the manual. However, 
this section fails to provide information linking the six sub-components of language 
(i.e. setting, audience, topic, etc.) to actual test items. There is no specific information 
detailing which items address some or all of the previously mentioned components of 
language. Criterion-related validity was evaluated in terms of concurrent validity. 
Evidence was collected by asking teachers to rate the pragmatic language skills of 30 
students who were given the TOPL (Phelps-Terasaki & Phelps-Gunn, 1992). The 
coefficient was .82 which supports the existence of concurrent validity; however, the 
result is limited by the small sample size (n=30). 
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Construct validity was evaluated in terms of six basic constructs: age 
differentiation, relationship of the TOPL to spoken language, relationship of the TOPL 
to school achievement, relationship of the TOPL to tests of mental ability, group 
differentiation of verbal abilities, and item validity. Support for construct validity is 
provided by age differentiation correlations. As students grow older their pragmatic 
language skills increase thus TOPL scores increase. To illustrate whether the TOPL 
relates to measures of spoken language, 30 students (grades k-2) were administered the 
language subtest of Screening Children for Related Early Educational Needs (SCREEN) 
(Hresko, Reid, Hammill, Ginsburg, & Baroody, 1988). The resulting coefficient was 
.70 which provides evidence for construct validity. To test the relationship between the 
TOPL and school achievement, 30 students (grade k-2) Math, Writing, and Reading 
subtests scores on the SCREEN were correlated with their TOPL scores. The resulting 
coefficients were .32, .39, and .55, respectively, suggesting they are related concepts. 
The relationship between the TOPL score and tests of mental ability was demonstrated 
by correlating the Scholastic Aptitude Scale Scores for 27 students with their TOPL 
scores. The resulting coefficient was .68, again providing support for construct 
validity; however no information was provided as to the grade level of the 27 students. 
All of these findings are limited by the small sample size. For group differentiation of 
verbal abilities, TOPL ^res of 24 students identified as having language disabilities 
were examined. The average TOPL quotient score was 87 which is below average 
thereby providing support for construct validity. With regards to item validity, the 
authors selected items which yielded at least a .30 point biserial (discriminating power) 
coefficient in discriminating between the item and the total test score. . Based on the 
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item selection procedures the TOPL test items do provide evidence of item validity. 
Overall, the validity findings are acceptable; however, they are limited by the 
utilization of a small sample and in some cases lack of information detailing the age or 
grade level of the students participating in the sample. 

Reviewers Comments 

Previous reviews of the TOPL by Ochoa (1995) and Wilkinson (1995) provide 
support for the utility of the TOPL while raising concerns about certain aspects of the 
instrument. Ochoa suggests more validity studies be conducted, and information on 
test-retest reliability may be beneficial. In addition, his review raises concerns about 
pictures and if2%. Picture 1f2Q depicts a record player which may go unrecognized 
by younger examinees since records have been replaced by Compact Discs. Picture 
1t2% depicts children building a snowman. This is a concern because some students, 
depending their geographic location, may not have seen snow and therefore cannot 
relate to the activity being demonstrated. In Wilkinson’s review, he raises questions 
about the interscorer reliability reported in the manual. Since the scorers were the test 
authors one would expect a high interscorer reliability coefficient. Wilkinson suggests 
field testing of interscorer reliability is warranted. Wilkinson also suggests a couple of 
additional precautionary directions be included in the manual: a) the need to administer 
all items, b) caution against accidentally administering a picture number instead of an 
item number-^since not all items have corresponding pictures. Both reviewers agree 
more information is needed relating actual test items to the six sub-components of 
pragmatic language stated by the authors. Finally, they agree that the TOPL is a useful 
tool for providing an in-depth screening of pragmatic language. 
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Summary Evaluation 

The TOPL is a useful tool in the assessment of pragmatic or social language 
skills of an individual. Specifically, it can be used for identifying persons with 
language difficulties, assessing a their strengths and weaknesses, or documenting 
progress of persons participating in an intervention. While the TOPL was developed 
with a nationally representative sample, test authors should consider additional validity 
studies utilizing larger and more diverse samples. The TOPL materials are well- 
written, easy to follow, and neatly packaged. Nevertheless, future versions should 
include a stand for the picture book as well as guidelines for an introduction for 
examiners to read before evaluation begins. Finally, better efforts should be made to 
demonstrate the relationship between test items and the six dimensions of pragmatic 
language. In sum, the TOPL works well in conjunction with other language assessment 
measures for diagnosis purposes, but it works best as a tool to provide a profile of a 
persons pragmatic or social language skills. 
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